Training Recurrent Neural Networks by Using Parallel Recursive Prediction Error Algorithm

As an alternative local and higher order algorithm, the parallel recursive prediction error algorithm (PRPE) is used to train the recurrent neural networks (RNNs). This algorithm uses a modiied form of the well-known recursive prediction error algorithm (RPE) such that the computation can be distributed into each node in the network. Therefore, the algorithm has a better trade-oo between computational cost and convergence time. Several examples of training the RNNs to perform time series prediction task are presented to demonstrate the superior convergence performance of the algorithm compared with the real time recurrent learning algorithm (RTRL).